Towards a Supervised Dyna-Q Application on a Robotic Manipulator

Having a robot that can learn a task shown to it is an challenge for robotic scientists and useful for nonengineers who want to interact with a robot. In this paper we address some of the difficulties one will have to overcome when developing such a system. Making a good data capturing of what is shown to the robot is one such problem, and one other is the curse of dimensionality that arises when we apply techniques successfully applied in mobile robotic to an articulated manipulator with more degrees-of-freedom (d.o.f.) than most mobile robots on wheels. We use reinforcement learning (RL) together with supervised learning (SL) to gain benefits from both paradigms. A key scientific challange is to adress the curse of dimensionality arising from the high dimensional state and action space in this application, which we propose to adress by combination of RL and SL. We also point out that one has to be careful when trying to obtain an agent that learns a task in as few trials as possible, since it might require much more computational time.